Evaluation of a digital FFQ using 24 h recalls as reference method, for assessment of habitual diet in women with South Asian origin in Norway

Objective: Dietary assessment tools should be designed for the target population. We developed an FFQ designed to assess diet in South Asian women in Norway. The study objective was to evaluate this FFQ using 24-h dietary recalls as reference method. Design: Approximately 3 weeks after the participants (n 40) had filled in the FFQ, the first of three non-consecutive 24-h dietary recalls was completed. The recalls were telephone-based, unannounced and performed by a trained dietitian, with 2–3 weeks between each interview. Setting: The DIASA 1 study, in Oslo, Norway. Participants: Women of South Asian ethnic origin participating in the DIASA 1 study were invited to participate in the evaluation study. Results: The WebFFQasia significantly overestimated the absolute intake of energy, protein, fat and carbohydrates compared with the 24-h dietary recalls. Absolute intakes of sugar, starch and fibre did not differ significantly between the methods. For energy percentages (E%), there were no significant differences, except for monounsaturated fat. Correlations were strong for E% from sugar and saturated fat and moderate for E% from fibre, carbohydrate, total fat and protein. Fourteen food groups out of twenty three were not significantly different compared with the reference method, and sixteen groups showed strong to moderate correlations. Conclusion: The WebFFQasia may be used to assess E% from habitual diet and can adequately estimate intakes and rank participants according to nutrient intake and main food categories at group level.

South Asian immigrants.Similarly, FFQ developed in their countries of origin do not reflect the changes in their diet after immigration (6) .Thus, there is a need for dietary assessment tools specifically designed for this population.
To address this issue, we recently evaluated the performance of a web-based FFQ designed for assessment of a traditional Norwegian diet, in South Asian women living in Norway, using three 24-h dietary recalls as a reference method (7) .The results showed this FFQ to adequately estimate the intakes of several food items and nutrients at the group level; however, estimates of protein, carbohydrates and dietary fat could be improved on.Based on the recipes gathered through the 24-h dietary recalls (7) , we developed a new version of the web-based FFQ specifically designed for individuals of South Asian origin living in Norway, the WebFFQasia.
The aim of the present study was to evaluate the performance of the WebFFQasia in a population of South Asian women living in Norway, using dietary information collected from three 24-h dietary recalls as reference method.

Study design and participants
The DIAbetes in South Asians (DIASA) 1 study included 179 women of South Asian (Pakistan, India, Sri Lanka and Bangladesh) and 108 women of Nordic (Norway, Sweden, Denmark, Finland and Iceland) origin, followed during specialist health care for gestational diabetes, 1-3 years before recruitment.The DIASA 1 study was performed between 1 September 2018 and 31 December 2021 at Oslo University Hospital, Akershus University Hospital and Drammen Hospital.A detailed description of the cohort has been published previously (8) .In brief, the DIASA 1 study investigates the impact of South Asian and Nordic ethnicity on the prevalence and characteristics of prediabetes and diabetes in women, 1-3 years after a pregnancy with gestational diabetes (8)(9)(10) .Eligible women received an invitation letter and a telephone invitation in their native language, and all participants signed study consent forms (8) .All women participating in DIASA 1 were invited to complete the WeFFQasia.From June 2021 to the middle of February 2022, women of South Asian ethnic origin who had completed the WebFFQasia were invited to participate in the evaluation study.Due to slow recruitment caused by the COVID-19 pandemic, we implemented retrospective recruitment, inviting participants who had already completed the WebFFQasia (>6 months in advance) from October 2021.These participants were asked to fill in the WebFFQasia questionnaire once more to participate.Exclusion criteria were non-Asian origin and a new pregnancy.We aimed at recruiting fifty participants (11) , based on the time frame and resources available in this project.

Background data
Age was calculated from the date of birth.South Asian origin was defined as both parents born in Pakistan, India, Bangladesh or Sri Lanka.Height and weight were measured at the study sites, according to international standards (12) and used to calculate the BMI.Height was measured to the nearest 0•1 cm by a fixed stadiometer, weight was measured to the nearest 0•1 kg by an electronic scale and BMI was calculated from the formula weight (kg)/height (m) 2 .The waist circumference was measured to the nearest 0•1 cm by a non-elastic measuring tape midway between the lower rib and the iliac crest, the participant standing and breathing normally, at exhalation.The hip circumference was measured at the trochanter region.Information about smoking and alcohol habits was collected through self-administered questionnaires in DIASA 1. Questions about education, living situation and work situation were asked during the evaluation study by the interviewer in the 24-h dietary recalls.

FFQ
The WebFFQasia is a web-based, self-administered, quantitative FFQ consisting of approximately 310 questions about habitual diet over the past year, including both South Asian and Norwegian dishes.The present WebFFQasia is a revised version of a previously validated original web-FFQ, designed to capture the habitual Norwegian diet, which includes 270 questions (13) .The changes and revisions made to the original web-FFQ, including the list of South Asian food items, were based on the results from a Master's thesis conducted in 2018 (7) .The Master's project evaluated the performance of the original web-FFQ in women of South Asian origin and concluded that revisions should be made to improve estimates of nutrient, energy and food intake.Specifically, questions about curries, chapati, idiyappam, dosa and South Asian desserts should be included.The original web-FFQ was thus revised according to these recommendations.When filling in the WebFFQasia at home, the participants indicated whether their habitual diet was generally Norwegian or South Asian.All who indicated the latter were then directed to additional twenty-three questions about specific South Asian food items before they continued with the rest of the WebFFQasia, consisting of the original web-FFQ.If the respondent indicated a mostly Norwegian diet, she was automatically led to the questions that constitute the original FFQ in WebFFQasia.The WebFFQasia also included questions about dietary supplements.The questions are largely organised in meals and associated food groups, including twelve questions about bread, thirty four about bread spreads, forty one about beverages (including hot and cold, with and without alcohol and drinking and mineral water), forty nine about dinners (vegetarian, meat and fish dinners), forty two about fruit, vegetables, berries, nuts and seeds, twenty four about desserts, sweets and salty snacks, in addition to questions about condiments, sources of fat and dietary supplements.Frequency alternatives range from never, times per month, times per week, to several times per day.Portion sizes are given in household measures, or in pictures of portions.The WebFFQasia has a dynamic design and may be used on all digital platforms, including mobile phones.The participants were therefore asked what digital platform they used when filling in the FFQ.The food and nutrient composition database and calculation system 'KostBeregningsSystem' (14) , database AE-22 (Department of Nutrition, University of Oslo) was used to estimate energy and nutrient intakes from the FFQ and 24-h dietary recalls.
The 24-h dietary recalls Three 24-h dietary recalls were conducted on three non-consecutive days, including two weekdays and one weekend day.The first interview was conducted at least, and approximately, 3 weeks after the participants had completed the WebFFQasia.The interviews were unannounced and telephone based and carried out by a trained dietitian, with 2-3 weeks between each interview.The interview followed a standard protocol for 24-h dietary recalls, developed at the Department of Nutrition, University of Oslo.In brief, according to protocol, the interviewer goes through all the meals the participants had the day before, twice and asks additional questions with regards to foods and beverages that we know from experience are easy to forget.On average, an interview would take approximately 30 min.All food items were coded directly into the KostBeregningsSystem database during the interview.To assist in the estimation of amounts, the participants used a booklet with pictures of portion sizes and sizes of plates, bowls, glasses and cups.Of all the interviews conducted, two individual interviews, with two different participants, were conducted without the booklet, because it was not available at the time, and the participants therefore provided their dietary information in household measures (half a plate, one cup, etc.) instead.The trained dietitian who performed the interviews was multilingual and conducted the interviews in Norwegian or the participant's preferred language (Urdu/Hindi/English).The dietitian was also familiar with the diet culture of South Asian populations in Norway.

Statistics
All statistical analyses were performed using the IBM SPSS statistical software package, versions 26 and 29.0.0.0.The statistical significance level was set to P < 0•05 for all analyses.Continuous variables were assessed for distribution and are presented accordingly.Categorical data are presented as numbers (%).We used paired-sample Student's t test or Wilcoxon's signed-rank test to assess differences between the dietary methods, according to distribution.Pearson's r and Spearman's ρ were used to explore correlation.Correlations were defined as poor when <0•30, moderate between 0•30 and 0•49 and strong when >0•50 (15) .To further evaluate the FFQ, cross-classification was used to compare quartiles of intake, Bland-Altman (BA) plots were used to assess systematic bias, (16) and drop-line graphs were used to further visualise individual differences between methods.

Study sample
Forty-two participants were included; however, one participant completed only one of the three 24-h dietary recalls and one recorded extreme and unrealistic levels of intake of several foods in the FFQ.Thus, data from forty participants were included in the analyses.Most participants were of Pakistani origin, followed by Indian and Sri Lankan origin (Table 1).Mean age was 35 years, range 25-42.

Intake of energy and energy-providing nutrients
Distributions of the absolute intake estimates of energy and energy-providing nutrients were skewed.The WebFFQasia significantly overestimated the absolute intake of energy, protein, fat and carbohydrates compared with the 24-h dietary recalls (Table 2).The absolute median differences between the methods, at group level, were as follows: 17 %, 15 %, 18 % and 13 % of the 24-h dietary recall estimated intakes, for energy, protein, fat and carbohydrates, respectively.Absolute intakes of sugar, starch and fibre did not differ significantly between the methods (Table 2).Correlations were strong for estimated intakes of saturated fat and moderate for all other energy-providing nutrients, except for total carbohydrate (Table 2).Cross-classification of participants into quartiles of intake showed that correct classifications ranged from 27 % for energy and n-3 fatty acids to 48 % for total fat.Misclassification into the opposite quartile ranged from 0 % for saturated fat to 10 % for total carbohydrates (Table 2).BA plots for absolute intakes of energy and total fat intakes are presented in Fig. 1, showing a trend towards increased differences with increasing mean intakes of energy, and wide limits of agreement.For total fat intake differences were evenly distributed above and below the mean intake.Intake of protein and carbohydrates showed the same increasing trends in difference as energy, while fibre, sugar and starch showed the same distributions of differences as total fat intake (Appendix Figures 1A-1E).
When estimating energy percentages (E%), there were no significant differences between the assessment methods for the energy-providing nutrients, except for monounsaturated fat, which the WebFFQasia significantly overestimated compared with the 24-h dietary recalls (Table 3).At a group level, the mean estimates from the FFQ differed with -0•3, 7•0, -4•7, -2•5 and 1•1 E% from the 24-h dietary recall estimates for protein, total fat, carbohydrates, sugar and fibre, respectively.Correlations were strong for E% from sugar and saturated fat and moderate for E% from fibre, carbohydrate, fat and protein.Poor correlations were seen for E% from mono-and polyunsaturated fat.The correct cross-classification ranged from 28 % for E% from polyunsaturated fat to 48 % for E% from total fat.Misclassification in the opposite quartile ranged from 0 % for saturated fat to 10 % for carbohydrates (Table 3).BA plots for the E% estimates showed broad limits of agreement and differences distributed randomly above and below the mean difference for E% from protein and total fat and increasing differences with increasing mean intake of E% from carbohydrates and sugar (Appendix Figures 1F-1I).
For total energy intake, the results from three analyses showed moderate to good evaluation (Spearman's correlation, cross-classification and BA plot), while group estimates showed poor evaluation (Wilcoxon's signed rank test and absolute difference).For absolute protein intake three analyses show moderate (Spearman's correlation, cross-classification and BA plot) and one showed poor (Wilcoxon's signed rank test) evaluation, while for protein E% two analyses showed moderate (Spearman's correlation and BA plot) and two showed good (Wilcoxon's signed rank test and cross-classification) evaluation.For both absolute intake and E% from total fat, cross-classifications showed good evaluations, and for E% from fat also group intake estimates   showed good evaluation.For total carbohydrate intake, three analyses showed poor evaluation and one (BA plot) showed moderate evaluation.For E% from carbohydrates, group estimate showed good evaluation (Wilcoxon's signed rank test), two analyses showed moderate evaluations (Spearman's correlation and BA plot), while the crossclassification showed poor evaluation.Individual differences between methods for energy, protein, fat and carbohydrates are further visualised in Appendix Figures 3A-3D.

Intake of vitamins and minerals
The WebFFQasia significantly overestimated the absolute intake of all micronutrients except vitamin D, Fe and Na, compared with the 24-h dietary recalls (Table 4).
Correlations were strong for thiamine, niacin, folic acid, vitamin B 12 , vitamin E, Zn and Se.There were moderate correlations for Mg, Na, iodine, phosphorus, Ca and vitamins A, B 3 , B 6 , C and D, and poor correlations were observed for Fe and potassium.Cross-classification into the correct quartile ranged from 30 % for Na to 50 % for vitamin B 6 .Misclassification in the opposite quartile ranged from 0 % for thiamine, Na, Se and iodine to 8 % for Fe, Ca and phosphorous (Table 4).A total of nine participants registered no use of dietary supplements in the WebFFQasia.Of these, four registered intake of dietary supplements and five did not register intake of dietary supplements in the 24 h recalls.

Intake of food and beverages
Intake of food and beverages in grams per day is presented in Table 5.The food intakes from the WebFFQasia were not significantly different from the 24-h dietary recall estimates for fourteen out of twenty-three food groups.Estimated intakes of cakes, cereals, rice and pasta, legumes, meat, egg, yoghurt, cheese, butter and margarine, sugar and sweets, all beverages and snacks showed no significant differences between methods.Intakes of bread, potato and potato products, vegetables (not including legumes), fruit and berries, nuts and seeds, fish and milk and cream were significantly overestimated by the WebFFQasia.Median intake of spices and herbs was significantly underestimated by the FFQ by 2 g/d.
For cakes, legumes, cheese and coffee, correlations between methods were strong.Of the remaining nineteen food groups, moderate correlations were observed for twelve and poor correlations for seven categories.Misclassification in the opposite quartile ranged from 0 % for coffee and snacks to 10 % for potato and potato products, legumes, fish and fish products and yoghurt (Table 5).
Intakes of all food groups except bread, other cereals, cakes and vegetables showed increasing differences between methods with increasing mean intake (Appendix Figures 2A-2H).Of these, fish, fruits and berries, juices, nuts and seeds showed tendencies towards increased overestimation with increased mean intake, tea and spices showed the opposite and the remaining food groups showed increasing differences randomly distributed above and below the average difference.
Out of the forty participants, nine answered 'no' to the initial question in the WebFFQasia of whether their diet included 'mostly South Asian dishes', and hence did not answer the questions about these specific dishes.However, according to the 24-h dietary recalls, six of these participants did register intake of South Asian dishes.There were no significant differences in energy intake between the participants who answered 'Yes' and 'No' to the 'mostly South Asian dishes' question (results not shown).

Discussion
The present study evaluated estimated intakes of energy, nutrients and food groups, assessed using an extended digital FFQ for women of South Asian origin living in Norway.Evaluated against the 24-h dietary recall reference method, the absolute intakes of energy and energy-providing nutrients were overestimated by the WebFFQasia.However, when calculated as E%, no significant differences were observed, except for monounsaturated fat.The WebFFQasia thus shows a good ability to assess E% at the group level.The ability to rank the participants according to E% was also good to adequate for E% from protein, total fat, total carbohydrates, sugar and fibre and poor for MUFA and PUFA.

Energy and energy-providing nutrients
The overall evaluation suggests clear trends of overestimation of energy from the WebFFQasia compared with the 24-h recalls.An earlier validation study showed that the energy intake estimated using the original web-FFQ did not differ significantly, at the group level, from total energy expenditure assessed using the doubly labelled water method (13) .In the same validation study, the estimated energy intake from 24-h dietary recalls was significantly underreported compared with the total energy expenditure assessments from the doubly labelled water.In the present study, the energy intake from the WebFFQasia was higher than the energy intake from the 24-h dietary recalls, as seen in the validation study from 2017 (13) .If the difference between energy intake from the WebFFQasia and the 24-h dietary recall was partly affected by possible underreporting of the 24-h dietary recalls, this is unknown but the results from the doubly labelled water study of 2017 may suggest this.However, as shown in both studies, the large individual variations in E% estimates from the two methods underline the importance of using the WebFFQasia to estimate intakes at the group level and that individual estimates from the WebFFQasia must be used with caution and a strong awareness of the uncertainty of the estimates at an individual level.Except for MUFA and PUFA, the evaluation of E% from the energy-providing nutrients showed, overall, good results, for both mean intakes at the group level and correlation and classification across quartiles of intake.This agrees with the results from both of the two earlier studies (7,13) , which showed overestimation of absolute intakes of macronutrients, whereas the E% from the energy-providing nutrients agreed with the reference methods.This therefore suggests that the present WebFFQasia can adequately assess E% at the group level in this population.Based on the correlation and cross-classification results, the WebFFQasia can rank the participants adequately for most macronutrients, except for MUFA, similar to the results from the evaluation study of the non-modified web-FFQ in South Asians (7) .

Vitamins and minerals
Most absolute intakes of vitamins and minerals were overestimated by the WebFFQasia.The exceptions were intakes of vitamin D, Fe and Na.This agrees partly with the earlier evaluation, which also found good agreement between the web-FFQ and the reference method for Fe and Na (7) .In the earlier study, there was also good agreement for the estimates of vitamin E and Se, which, in the present evaluation, showed significant differences between methods.These differences in results may have random causes or be explained by the added questions about South Asian dishes where vegetable oils, sources of vitamin E in the diet, are important ingredients and also used widely for cooking.Variations between the amounts of vegetable oils used in the standard recipes of the food composition database and individual variations in use among the participants while cooking may have influenced the results.
In the present evaluation study, good ranking ability was observed for vitamin B 1 , vitamin B 2 , folic acid, vitamin B 12 , vitamin E, Zn and Se, and adequate ranking was seen for the remaining investigated micronutrients, except potassium and Fe.Thus, the WebFFQasia may be used to rank participants according to intakes of vitamin and minerals; however, results for Fe and potassium should be used with caution.

Food groups
In the present evaluation, the WebFFQasia showed a good ability to estimate absolute intake of food groups at the group level, for fourteen of the twenty-three food groups, whereas intakes of bread, milk and cream, fruit, berries, nuts, potatoes, vegetables, not including legumes, and fish were all overestimated at the group level compared with the reference method.This agrees partly with the findings of Medin et al. also showing overestimation of potatoes, fruit and vegetables, fish, milk and cream in the original web-FFQ (13) .
The correlations and cross-classification results showed moderate-to-good agreement, and the WebFFQasia may thus be used to rank participants according to the main food groups, but should, however, be used with some caution for estimates of potato, potato products, vegetables (not legumes) and snack intakes.

Study population
In the present study with women of South Asian origin in Norway, we met challenges related to recruitment and data collection.Quay and colleagues (6) identified factors that could facilitate recruitment and data collection in dietary surveys among South Asians in the UK, including culturespecific assessment tools, visual aid reinforcement and the involvement of key community members and translators.In the present study, we applied an online, webbased FFQ that was designed to give better coverage of South Asian diets and common South Asian dishes.We used visual aids with regard to portion sizes and a dietitian who spoke Urdu and Hindi and was familiar with South Asian diet culture conducted the 24-h dietary recalls.A total of 68 % of the study population had education at university level.This is a higher share than observed for the general population of women with South Asian origin in Norway, which is approximately 38 % (statistics Norway, accessed November 2023, ssb.no/en/statbank/), and this may influence the results.

Strengths and limitations
The strengths of the study include the use of culturespecific assessment tools, 24-h dietary recalls being conducted in Urdu or Hindi if necessary and the interviewer having knowledge of Asian diet and culture.These measures may have reduced the participation burden, as pointed out by Quay and colleagues (6) .
All self-reporting dietary assessment methods are prone to measurement errors.Misreporting of food and energy intake using self-reported dietary assessment methods has been extensively reported (17)(18)(19)(20)(21) .The inherent limitations of the closed food lists, the abstraction of frequencies and the standard portion sizes of an FFQ make it a difficult cognitive task for respondents and challenge their memory (22) .Underreporting of food intake when diet is assessed using 24-h recalls has been seen to be associated with both observational and reporting effects (20) .Also, due to the design of 24-h recalls, the method may be prone to underestimate intake of foods that are eaten infrequently or seldom.In an evaluation study, such as the present one, evaluation of an FFQ designed for assessing average habitual diet over the last year, with frequency alternatives including infrequent intakes, using 24-h recalls as the reference method, may result in a more uncertain evaluation of infrequently and rarely eaten food items and the associated micronutrient intakes.Some of the poor agreement between the methods could be due partly to 24-h dietary recalls not capturing less frequent intakes, compared with the FFQ which covers average intake over the last year.This may explain the poor agreement observed between estimated intakes of fish, a food group many eat infrequently and seldom.Hence, disagreement between the methods may result from errors in both methods and suggests that the assessment of certain seldom eaten food items should be evaluated against other dietary assessment methods that capture rare intakes better than 24-h dietary recalls (22) .
The small sample size is a limitation of this study, and a study sample of at least fifty, preferably larger is desirable (11) .This will have affected the analyses and the outcomes of the study, making the results less robust.However, the evaluation study was performed in a subgroup of the main study population, which is a strength as age, ethnic group, gender and health status of the population also may affect the results of an evaluation study (11) .

Conclusions
The present evaluation of the WebFFQasia, an FFQ designed to assess the habitual diet of South Asian women living in Norway, showed that it may be used to assess E% from habitual diet and can adequately estimate intakes and rank participants according to the intakes of the nutrients and main food categories at the group level.The dietary assessment tool is designed for group-level estimates and should not be used for individual dietary assessments due to the high level of individual measurement errors.Future studies applying the WebFFQasia for dietary assessment may reflect on a potential tendency of overreporting absolute intakes of fruit, vegetables and fish.

Fig. 1
Fig.1Bland-Altman plot of intake of (a) energy, (b) total fat, from the FFQ and the 24-h recalls.Mean intake on the x-axis (mean of FFQ and 24-h recalls) against the difference in intake (FFQ -24-h recalls) on the y-axis, in kJ/d.Dotted lines are limits of agreement (mean difference ± SD × 1•96)

Table 1
Anthropometric and socio-demographic background of participants (n 40) Height and weight were measured at the study clinic.Position ≥50 %, employment part time, working hours 50 % or more.

Table 2
Estimated absolute intakes of energy and energy providing nutrients from the FFQ and the 24-h recalls (n 40)

Table 3
Estimated energy percentage (E%) from energy-providing nutrients, from the FFQ and 24-hour recalls (n 40)

Table 4
Micronutrient intake estimated from the FFQ and the 24-hour recalls (n 40)

Table 5
Estimated intakes of food and beverages from the FFQ and 24-h recalls (g/d)